There exist well established and effective methods for compressing photographic images such as DCT-based JPEG image compression (JPEG). However JPEG is not suited to compressing images which contain sharp edges such as occur in images containing text or object-based computer graphics. For such image content the size or quality of JPEG compressed images leaves a lot of room for improvement.
Many viewed and printed images contain both photographic image data and computer-generated text or graphics. There is a need in computer and printing systems to improve the compression of such mixed content images.
One approach to compressing mixed content images is to segment the image into regions which are separately coded. By performing the segmentation so that boundaries between segmented regions are aligned with sharp edges, JPEG can be used to code image data without those sharp edges. The Mixed Raster Content (MRC) standard, ITU-T Recommendation T.44 (01/2005) specifies a compressed representation for this approach to compression of mixed content images. In the MRC representation, an image is segmented so that each decompressed image pixel has its value selected from one of a set of image content ‘layers’ with the selection being specified by one or more bi-level mask layers. The MRC standard does not specify how to perform the segmentation which can be a difficult and compute-intensive operation. Segmentation for MRC image compression is often simplified by adopting a block-based approach.
The MRC compression model builds on well established compression algorithms by coding each MRC layer to suit the layer's content using a standard raster image coding method operating on a rectangular region of pixels. One of the important advantages of the MRC model is that segmentation can allow use of transform coding without requiring transform coding to deal with some of the extended sharp edges in an input image. This is because, when an MRC segmentation boundary follows a sharp edge of an image, decompression of an MRC compressed representation takes data from each side of the edge from separate image content layers and consequently there is no requirement to represent the sharp edge in either of the image content layers. However, this sharp edge removal has a computational and compression overhead because if the edge is not at the rectangular boundary of an image content layer then it is necessary to fill in and code image values, which are not used in the decompressed image, at the “don't care” pixel positions of the image content layers. This process of filling in image values at the “don't care” pixel positions is sometimes known as ‘back filling’ or ‘plane filling’. To minimise transform coding of sharp edges, such fill values of an image content layer must be chosen to be similar to image values of nearby used pixel positions of the same content layer.
There exist other methods of mixed content image compression which also follow the segmentation approach and which identify regions of constant colour for lossless compression and remaining regions for compression using JPEG. These methods compress image blocks by lossless compression or JPEG compression or by a combination of lossless and JPEG compression. For those blocks applied to JPEG compression which also contain pixels extracted for lossless coding, the extracted pixels are assigned new image values equal to the average colour of un-extracted pixels. This tends to reduce the presence of sharp edges and improve JPEG compression performance.
Another approach to compressing mixed content images is known as “sketch based coding”, as introduced in the paper “Sketch Based Coding of Grey Level Images”, S. Carlsson, Signal Processing, 15, 57-83 (1988). In this approach the values of pixels on both sides of visually significant edges are coded. A “sketch image”, which mimics the discontinuities at those edges but which is smooth elsewhere, is constructed which has image values for pixels between coded edges calculated as an interpolation of image values at coded edges. The residual or difference between the original image and the sketch image, now largely free of extended sharp edges, can be coded effectively by transform image compression methods such as JPEG. The calculation of the sketch image is a multi-scale and iterative interpolation process which can be time-consuming and require large memory resources.
A similar approach to image compression is to express an image as the sum of a primary image with strong edges but elsewhere smooth, a smooth image, and a texture image.
Previous methods of extracting edge data from an image for calculating an approximate image, such as those described by Carlsson, suffer from difficulties in linking isolated edges into edge runs, with the result that coding of edge data as edge runs is unable to exploit the data redundancy of edges well. This in turn results in reduced compaction and reduced quality of the approximate image.